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ABSTRACT 


ELEMENTS  OF  SUCCESS  FOR  U.S.  AIR  FORCE 
MILITARY  CONSTRUCTION  PROJECTS 

by 

STEVEN  ANDREW  DAVIS,  MASTER  OF  SCIENCE  IN  ENGINEERING 
THE  UNIVERSITY  OF  TEXAS  AT  AUSTIN,  1993 
SUPERVISOR;  RICHARD  L.  TUCKER 

A  wide  range  of  U.S.  Air  Force  construction  projects  completed  in  the  continental 
United  States  were  evaluated  to  (1)  mea^e  the  effectiveness  of  &ctors  impacting  the 
constniction  process,  (2)  measure  the  key  indicators  of  project  success,  and  (3) 
determine  the  relationship-between  these  input  factors  and  indicators  of  success.  Input 
Victors  and  measures  of  success  were  devdoped  from  previous  research  and  applied  to 
the  militsry  construction  environment.  The  four  key  measures  proposed  to  indicate 
project  success  included:  (1)  project  quality  and  performance,  (2)  cost  variability,  (3) 
schedule  performance,  and  (4)  safety.  The  factors  assumed  to  most  significantly 
impact  success  were:  (1)  derign  basis,  (2)  qualified  key  leaders,  (3)  compatible 
objectives,  (4)  wdl  defined  responsibilities,  (5)  control  ^sterns,  and  (6)  project 
environment. 

An  objectives  matrix  approach  was  used  to  combine  input  factor  criteria  and  create 
an  evaluation  index  for  each  project.  The  same  process  was  used  for  the  output 
measures  criteria.  Liput  and  output  indexes  were  plotted  against  each  other  and 
analyzed  to  determine  the  degree  of  correlation. 

Relatively  high  indexes  indicated  good  project  management  in  general.  In  addition, 
data  indicated  a  relatively  strong  linear  correlation  between  input  and  output  indexes. 
Finally,  the  logical  cause-and-effect  relationship  suggests  that  it  is  possible  to  predict 
project  success  based  upon  input  data.  Therefore,  the  objectives  matrix  evaluation  of 
input  factors  would  be  a  good  predictor  of  ultimate  project  success. 
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1.  Introduction 


1.1  Purpose 

This  research  is  deigned  to  determine  the  key  fiictors  which  have  a  critical  impact 
on  the  success  of  a  constructed  project.  The  goal  is  to  identify  factors  assumed  to 
impact  construction  project  success,  and  measure  resulting  success  in  a  way  that  (1) 
geno’ates  an  input  factor  index  value  by  evaluating  and  combining  the  input  factors 
identified,  and  (2)  generates  an  output  measures  index  value  by  determining  and 
combining  the  output  measures  identified,  so  that  the  two  indexes  may  be  plotted 
against  each  other  for  a  number  of  projects  to  determine  whether  they  have  a 
relationship. 

The  research  hypothesis  is  that  weU  accomplished  input  factors  should  ensure 
ultimate  project  success.  If  input  &ctor  measures  are  related  to  project  success,  they 
could  be  used  to  predict  project  outcon^.  If  the  outcome  is  not  predicted  as  positive, 
input  measures  should  give  project  managers  an  idea  of  which  areas  to  correct,  in 
order  to  improve  that  outcome. 


1.2  Scope 

Twenty-two  U.S.  Air  Force  construction  projects  have  been  evaluated  to  (1) 
measure  the  effectiveness  of  inputs  to  the  construction  process,  and  (2)  measure  the 
indicators  of  project  success.  The  projects  represent  a  wide  range  of  construction 
types  for  the  U.S.  Air  Force  in  the  continental  United  States.  The  projects  are  listed  in 
Chapter  3.  Major  inputs  and  measures  of  success  were  suggested  by  previous  research 
and  validated  by  a  panel  of  nulitary  construction  project  managers.  The  proposed 
measures  of  project  success  include:  (1)  project  performance,  (2)  cost  variability,  (3) 
schedule  performance,  and  (4)  safety  performance.  Factors  assumed  to  impact  success 
are:  (1)  deagn  basis,  (2)  qualified  key  leaders,  (3)  compatible  objectives,  (4)  well 
defined  responsibilities,  (5)  control  ^ems,  and  (6)  project  environment. 

Project  data  were  collected  by  surv^ng  military  construction  project  managers 
fi'om  the  U.S.  Air  Force  Center  for  Environmental  Excellence  (AFCEE)  at  Brooks  Air 
Force  Base,  Texas,  and  by  obtaining  compiled  project  data  from  the  AFCEE 
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Programming,  Design,  and  Construction  (PDC)  project  database  for  recently 
completed  projects. 

An  objectives  matrix  approach  was  used  to  combine  criteria  and  create  both  an 
input  factors  evaluation  and  an  output  measures  index  for  each  project.  An  output 
sensitivity  analysis  was  conducted  to  determine  the  emphasis  of  output  criteria 
weighting  on  the  output  measures  index  for  each  project.  Input  factors  and  output 
measures  indexes  were  compared  and  plotted  and  a  regression  analysis  performed  to 
determine  their  relationship. 


2.  Background 


2.1  Importance  of  Construction  Metrics 

Metrics  are  measures  of  how  well  we  do  things.  Good  metrics  (1)  are  meaningful 
to  the  customer;  (2)  tell  how  well  organizational  goals  and  objectives  are  met;  (3)  are 
simple,  logical,  and  repeatable;  (4)  show  a  trend;  (S)  are  defined  unambiguously;  (6) 
are  economical  to  collect;  (7)  are  timely;  and  (8)  drive  the  "appropriate  action."  Initial 
and  ongoing  metrics  are  necessary  in  improving  any  process.  Initial  metrics  evaluate 
past  performance  and  determine  how  effective  efforts  have  been  in  achieving 
objectives.  Ongoing  metrics  measure  the  results  of  improvement  efforts,  point  out 
subpar  performance  and  problem  areas,  and  most  importantly,  drive  the  "appropriate 
action." 

A  construction  measurement  system,  beginning  with  a  project  database  identifying 
project  performance  and  corresponding  input  efforts,  can  be  a  powerful  management 
tool.  Construction  metrics  can  be  used  to  compare  the  progress  of  ongoing  projects 
against  past  projects  and  flag  shortfalls  and  problems  to  management  attention. 
Compiled  "lessons  learned"  can  suggest  effective  actions  in  alleviating  problems  or  in 
optimizing  performance  by  identifying  what  has  worked  well  and  what  has  not  worked 
well  for  past  projects.  Finally,  metrics  can  determine  how  effective  improvement  and 
iimovation  efforts  are  in  achieving  project  access  by  evaluating  the  "innovative" 
project  against  the  "standard"  project. 

2.2  Measures  of  Success 

Elementally,  project  success  means  meeting  or  exceeding  project  objectives  set 
early  in  a  project.  Carrying  this  one  step  further,  project  success  is  incomplete  unless 
it  leads  to  the  success  of  larger  organizational  objectives.  Therefore,  success  can  best 
be  evaluated  by  measuring  the  level  of  accomplishment  of  a  project's  unique 
objectives,  and  the  objectives  of  the  larger  organization  relating  to  that  project. 

The  best  indicator  of  project  success,  customer  satisfaction,  is  usually  not 
completely  available  until  the  facility  has  been  in  service,  possibly  not  until  after  it  has 
served  its  useful  life.  However,  it  is  very  difficult  to  compile  complete  data  for  these 
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projects,  since  ail  of  the  people  familiar  with  them  are  usually  not  available.  In 
addition,  long>term  results  don't  provide  the  immediate  feedback  necessary  to  improve 
the  process  for  upcoming  projects.  Th^efore,  the  most  practical  time  to  get  data  is 
probably  at  or  near  the  end  of  a  project. 

A  Construction  Industry  Institute  (CII)  study  identified  design  outcome  parameters 
focused  primarily  on  end-of-project  or  post-project  results.  The  most  significant 
parameters  were  idoitified  as  follows;  (1)  final  project  schedule,  (2)  constructability, 
(3)  quality  of  design,  (4)  final  project  cost,  (S)  plant  start-up,  (6)  plant  performance, 
and  (7)  safety  (Cn  Publication  8-2,  87).  The  final  project  schedule  parameter 
identifies  how  well  the  schedule  objectives  of  the  project  were  met.  Constructability  is 
the  integration  of  construction  knowledge  and  experience  in  planning,  engineering, 
procurement,  and  field  operations  to  achieve  project  objectives.  Quality  of  design 
evaluated  design  documents,  including  drawings  and  specifications  for  meeting  the 
owner's  specified  requirements,  completeness,  correctness,  clarity  and  timeliness. 

Final  project  cost  is  how  well  total  fadlity  cost,  including  design  and  construction,  met 
budget  requirements.  Plant  start-up  is  the  effort  to  bring  a  plant  into  operation  and 
prepare  it  for  occupancy.  It  consists  of  a  series  of  checks  to  ensure  it  will  operate  as 
designed.  Performance  relates  to  meeting  operating  requirements  throughout  the  life 
of  a  facility.  These  will  vary  for  each  project  depending  on  the  owner’s  objectives,  but 
may  include  flexibility,  reliability,  edacity,  economy  of  maintenance,  and  aesthetics. 
Safety  evaluates  the  accident  record  of  construction  and  the  safety  problems  of  the 
completed  facility. 

These  seven  parameters  were  conadered  for  applicability  to  different  projects  for 
this  research.  A  discussion  of  selection  of  the  final  parameters  is  in  chapter  4. 

2.3  Factors  Impacting  Success 

The  Project  Organization  Task  Force  of  the  Construction  Industry  Institute  (CII) 
identified  the  following  elements  that  determine  project  success;  (1)  key  leaders,  (2) 
compatible  objectives,  (3)  design  baas,  (4)  project  strategy,  (5)  roles  and 
responsibilities,  (6)  information  systems,  (7)  qualified  people,  and  (8)  woridng 
relationships  (Carroll  89).  These  factors  vary  in  importance  depending  on  the  client 
and  the  type  of  fiicility  being  constructed. 
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Key  leaders:  the  project  executive,  the  owner's  project  manager,  and  the  general 
superintendent,  are  mainly  concerned  with  work  flow  rather  than  specific  work 
packages.  The  project  executive  determines  the  owner's  objectives  and  establishes 
direction  for  the  project.  The  owner's  project  manager  takes  the  direction  of  the 
project  executive  in  managing  de»gn,  procurement,  and  construction  actiwties.  The 
general  superintendent  plans  and  coordinates  construction  field  activities. 

The  owner  determines  and  prioritizes  project  objectives  and  communicates  them 
through  contracts  and  planning  documents.  Differences  and  conflicts  should  be 
discussed  and  resolved  to  eliminate  work  barriers. 

The  design  basis  is  the  compiled  input  for  the  design  process.  Design  basis  includes 
specific  processes  and  facility  requirements,  project  scope,  site  investigations,  design 
standards,  necessary  reviews  and  analyses,  etc. 

The  project  strategy  is  the  overall  plan  to  accomplish  a  project,  including  a 
contracting  strategy,  a  facilities  plan,  an  external  map,  and  a  project  schedule.  A 
contracting  strategy  includes  an  explanation  of  organization  fiinctions,  as  well  as  an 
acquisition  plan  identifying  potential  sources  of  materials  and  services,  and  the  means 
of  obtaining  them.  A  facilities  plan  defines  the  enviroiunent  in  which  work  will  be 
performed,  including  work  locations,  temporary  facilities,  logistics,  and 
communications.  An  external  map  identifies  the  entities  which  influence  the  project, 
even  though  they  may  not  have  a  direct  contractual  relationship.  The  project  schedule 
is  the  timing  of  engineering,  procurement,  and  construction  tasks.  The  schedule  must 
include  input  firom  all  significant  parties:  the  owner,  designer,  constructor  and  major 
vendors. 

Contractual  agreements  establish  the  roles  and  responsibilities  of  parties.  Th^ 
should  be  written  to  promote  cooperation  and  avoid  advnwial  relationships. 
Authority  should  go  with  responsibility  and  risks  should  be  assigned  to  those  who  can 
best  control  them.  Contracts  should  encourage  the  participation  of  qualified 
organizations  and  reflect  the  agreement  between  them.  Special  provisions  should  be 
used  to  state  expectations,  including  partidpation  in  the  management  process. 

Information  systems  gather,  compile,  and  distribute  data  for  scheduling,  materials, 
equipment,  and  facilities  management. 
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Each  function  must  be  staffed  with  qualified  personnel.  The  project  manager  for 
each  organization  and  key  leaders  are  the  most  important.  Qualified  people  can  work 
around  shortfalls  in  the  formal  organization. 

Effective  communication  depends  upon  mutual  cooperation.  Individuals  involved 
in  project  processes  should  be  part  of  establishing  flow  charts  and  project-wide 
conununication  systems.  Participants  better  understand  the  needs  of  other 
organizations  and  will  have  a  personal  commitment  to  make  their  system  work. 


2.3  The  U.S.  Air  Force  Military  Construction  (MILCON)  Process 

The  military  construction  process  is  much  like  that  of  any  corporation. 
Requirements  are  identified,  a  project  is  planned,  authorized,  and  constructed  to  meet 
those  requirements,  and  the  resulting  facility  is  operated.  The  overall  process  is  shown 
in  Figure  2.1. 

The  "typical”  MILCON  project  starts  when  a  uang  activity  identifies  a  requirement 
to  the  Base  Civil  Engineer  (BCE).  The  BCE,  through  the  civil  en^eering  operations 
and  contract  programming  branch,  determines  whether  the  work  will  require  a 
MILCON  project.  MILCON  projects  are  usually  reserved  for  more  complex 
construction  activities  costing  more  than  $200,000.  The  project  must  be  validated  and 
approved  by  a  base  level  fitcUities  board  representing  the  major  base  using  agencies, 
cm!  engineering,  and  the  wing  commander.  If  it  passes  this  test,  it  is  submitted  to  the 
Major  Command  (MAJCOM),  which  then  must  approve  and  submit  it  to  the 
Headquarters  US  Air  Force  (HQ  USAF)  fiicilities  panel  for  design  approval.  Once 
validated,  design  is  authorized  and  a  design  instruction  is  issued  by  AF/LEE,  the  Air 
Force  Directorate  of  Engineering.  The  project  is  placed  in  the  Program  Objective 
Memorandum  (POM)  for  review  by  the  Office  of  the  Secretary  of  Defense  (OSD)  and 
the  Joint  Chieft  of  Staff  (JCS).  Design  starts  while  the  project,  along  with  other 
Department  of  Defense  projects  in  the  POM,  is  being  reviewed. 

Ideally,  design  reaches  35%  at  the  start  of  the  budget  process.  Approved  projects 
are  submitted  in  the  Air  Force  Budget  Estimate  Submittal  (BES)  to  the  Deputy 
Assistant  Secretaiy  of  Defense  (Programming/Budget)  (DASD  (P/B)).  Hearings  are 
held  Avith  OSD  and  the  Office  of  Management  and  Budget  (0MB)  and  result  in 
Program  Budget  Decisions  (PBD).  The  PBDs  are  submitted  by  DASD  (P/B)  for  the 
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Presidential  Budget  with  Justification  Books  sent  to  Congress.  Budgets  are  reviewed 
and  approved  by  the  Armed  Services  and  Appropriations  Committees  of  both  the 
House  and  Senate.  Discrepancies  are  worked  out  by  a  joint  Conference  Committee. 
Projects  are  approved  and  funded  when  the  President  signs  the  Appropriation  and 
Authorization  Bills  into  law.  Constmction  funding  is  requested  by  the  Military 
Services  Headquarters  through  OSD  and  0MB  and  then  provided  to  the  construction 
agent. 

If  the  project  was  not  deleted  by  the  joint  Conference  Committee,  HQ  USAF 
authorizes  the  Air  Force  Construction  Manager  (CM)  and  the  applicable  construction 
agent  (CA),  either  the  Army  Corps  of  Engineers  (COE)  or  the  Navy  Facilities 
Engineering  Command  (NAVFAC),  to  advertise  for  construction.  Opening  of 
construction  bids  must  wait  until  the  bill  becomes  public  law.  If  bids  exceed  the 
programmed  amount  (PA),  the  CA  can  request  additional  fimding  from  the  MAJCOM 
(if  less  than  10%),  or  reduce  cost  and  project  scope  (if  more  than  10%).  After 
contract  award,  and  pre-construction  conference,  the  contractor  is  given  a  notice  to 
proceed.  Construction  begins  after  approval  of  required  submittals. 

The  contractor  is  required  to  warrant  workmanship  and  material  for  one  year.  The 
project  manager  should  perform  a  post  occupancy  evaluation  after  nine  to  eleven 
months  of  beneficial  occupancy  to  identify  any  defective  work.  Deficiencies  are 
reported  to  the  CA  for  correction  by  the  contractor.  The  project  manager  should  also 
document  any  problems  and  pass  this  information  to  the  design  agent  and  other  project 
managers  to  review  for  similar  projects. 


Figure  2.1;  MBLCON  Process  Flow  Chart 


3.  Study  Methodology 


3.1  The  Process 

Research  consists  of  three  basic  phases:  research  definition,  research  design  and 
testing,  and  data  collection  and  analysis  (Broaddus  91).  In  definition,  a  research 
objective  and  possible  input  and  output  measures  are  defined,  and  potential  projects 
and  points  of  contact  are  identified.  In  design  and  testing,  initial  input  and  output 
matrices  are  developed,  and  a  data  survey  is  designed  and  fine  tuned  with  the 
collection  and  analysis  of  pilot  data,  to  capture  both  input  and  output  data.  In  data 
collection  and  analysis,  data  are  collected  and  project  input  and  output  indices  are 
calculated  with  objectives  matrices,  corresponding  indices  are  plotted  (input  index 
plotted  against  the  corresponding  output  index  for  each  project),  resulting  data  points 
are  analyzed  and  conclusions  are  developed. 

3.2  Definition  of  Inputs  and  Outputs 

Beginning  with  measures  of  success  and  input  factors  compiled  from  existing 
research  (Carroll  89;  Broaddus  91),  a  panel  of  project  managers  from  the  construction 
management  directorate  of  the  U.S.  Air  Force  Center  for  Environmental  Excellence 
(AFCEE)  was  surveyed  to  determine  and  weight  appropriate  inputs  and  outputs  to 
measure  Air  Force  construction  projects  (see  Table  3. 1  for  listing  of  panel  members). 
The  researcher  identified  project  performance,  cost  variability,  schedule  performance, 
and  safety  as  appropriate  measures  of  success  from  a  literature  review  (Broaddus  91). 
The  researcher  also  proposed  compatible  objectives,  qualified  key  leaders,  design 
basis,  control  systems,  project  envdronment,  ana  well  defined  responabilities  as 
appropriate  factors  impacting  success  fi’om  the  literature  review  (Carroll  89). 

The  surveyed  panel  adopted  the  researcher's  proposed  significant  input  factors  as 
follows:  (1)  design  basis,  (2)  qualified  key  leaders,  (3)  compatible  objectives,  (4)  well 
defined  responsibilities,  (S)  control  systems,  and  (6)  project  environment. 

Design  basis  is  the  compiled  input  for  the  design  process.  It  includes  specific 
processes  and  facility  requirements,  project  scope,  site  investigations,  design 
standards,  necessary  reviews  and  analyses,  etc. 
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Table  3.1:  AFCEE  Panel  Members 

Namg 

Tanya  Banks 

Deputy,  Medical  Division 

Dave  Cole 

Project  Manager,  Medical  Divirion 

Gene  Deramus 

Designer,  Planning  &  En^neering  Division* 

Bob  Lester 

Project  Manager,  Space  Division 

Randy  Lierly 

Project  Manager,  Space  Division 

Cleo  Walton 

Project  Manager,  Medical  Division,  Air  University 

*  former  project  manager 

Key  leaders,  especially  the  project  manager,  and  general  superintendent,  can  make 
or  break  a  project.  The  project  manager  is  responsible  for  ensuring  adequate 
objectives  have  been  set  and  that  those  objectives  are  met  by  managing  design, 
procurement,  and  construction  activities.  The  general  superintendent,  by  how  well  he 
plans  and  coordinates  construction  field  activities,  has  a  tremendous  impact  on  actual 
construction. 

Efforts  to  define,  prioritize,  and  communicate  project  objectives  through  contracts 
and  planning  documents,  both  at  the  be^nning  of  and  throughout  a  project,  are 
incredibly  important  to  project  success.  Weil  established  and  honest  objectives  open 
communication  and  can  address  differences  and  conflicts  early  to  eliminate  serious 
potential  problems  later  in  a  project.  The  level  of  agreement  on  objectives  also  sets 
the  stage  for  the  project  environment. 

The  level  of  effort  spent  defining  responsibilities  helps  players  understand  their 
roles  in  accomplishing  project  objectives.  This  effort  can  help  assign  risks  to  those 
who  can  best  control  them  and  authority  and  responribility  to  the  most  qualified 
people.  Establishing  clear  responsibilities  promotes  an  environment  of  cooperation 
and  reduces  adversarial  relationships. 

Control  systems  gather,  compile,  and  distribute  data  for  scheduling,  materials, 
equipment,  and  facilities  management.  These  systems  make  it  possible  to  effectively 
manage  the  complexity  of  a  construction  project. 
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Perhaps  the  best  predictor  of  overall  project  success,  project  environment  includes 
everything  beyond  the  control  of  the  project  manager.  Other  factors  already 
mentioned  have  an  impact  on  environment.  The  degree  of  agreement  on  objectives, 
definition  of  responsibilities,  and  level  of  effective  communication  and  mutual 
cooperation  can  impact  the  project  environment.  It  should  be  noted  that  the  panel 
adopted  the  proposed  input  factors  even  though  other  &ctors  may  also  significantly 
impact  project  success. 

The  panel  adopted  the  proposed  significant  measures  of  success  as  follows;  (1) 
project  performance,  (2)  schedule  performance,  (3)  project  cost  variance,  and  (4) 
safety.  Project  performance  is  how  well  project  objectives  are  met  and  how  well 
operating  requirements  throughout  the  life  of  the  facility  will  be  met.  These  will  vary 
for  each  project,  depending  on  objectives,  but  may  include  aesthetics,  flexibility, 
reliability,  capacity,  and  economy  of  mmtenance.  The  project  schedule  parameter 
identifies  how  well  project  schedule  objectives  have  been  met.  Project  cost  va  nee  is 
the  difference  between  final  project  cost  and  the  estimate  of  project  cost  at  project 
award  divided  by  the  cost  estimate  at  award.  Safety  performance  is  an  evaluation  of 
the  mishap  and  accident  record  during  construction  and  the  safety  problems  of  the 
completed  facility.  For  this  research  effort,  accurate  safety  statistics  were  not  readily 
available,  and  safety  performance  was  based  on  the  project  manager’s  rating  firom  the 
project  manager  survey.  Even  though  these  output  measures  are  widely  used  and 
generally  accepted,  there  may  be  other  measures  which  are  equally  appropriate  to 
measure  construction  success. 


3.3  Project  Selection  Considerations 

Ideally,  large,  recently  completed  projects  provdde  the  most  significant  data.  Large 
projects  tend  to  be  more  complex  and,  therefore  a  better  test  to  determine  the  most 
significant  input  criteria.  Recent  projects  are  better  for  collecting  subjective  data  since 
key  project  participants  are  more  likely  to  be  available  and  data  is  more  likely  to  be 
remembered  for  recent  projects. 

Projects  were  selected  fi*om  the  AFCEE  Master  Project  Listing  of  Military 
Construction  Program  (MBLCON)  projects.  For  this  research,  nearly  or  recently 
completed  projects,  most  with  final  construction  working  estimates  (CWE)  greater 
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than  $2  M,  were  considered.  Smaller  projects  were  included  when  data  became  more 
difficult  to  obtain.  Projects  were  not  selected  if  project  managers  had  recently  taken 
over  management  duties  and  were  unfamiliar  with  either  input  or  output  information. 
The  project  listing  is  shown  in  Table  3.2. 

TABLE  3.2:  PROJECT  LISTING 


Location 

Proiect  Title 

CWE 

($000) 

£M 

ALTUSAFB.OK 

FIELD  TRAINING  FACILITY 

4,500 

Steck 

ANDREWS  AFB,  MD 

ALT  MED  CTR  CLINIC  ANNX 

2,537 

Hughes 

BEALE  AFB.  CA 

ADAL  POWER  PLANT 

9,073 

Steck 

BUCKLEY  ANGB,  CO 

UPGRADE  EMRGNCY  POWER 

7,530 

Hodges 

GUNTER  AFB,  AL 

STUDENT  DORMITORY 

3,014 

Udall 

GUNTER  AFB,  AL 

^D  TO  SOFTWARE  DEV 

6,342 

hfoore 

HOLLOMAN  AFB,  NM 

CONTROL  TOWER 

3,000 

Lester 

KIRTLANDAFB,NM 

BC-AFISC  HEADQUARIERS 

13,011 

Steck 

LACKLAND  ANX,TX 

ADAL  OPERATIONS  FACILITY 

1,442 

Castamore 

LITTLE  ROCK  AFB,  AR 

HOSPITAL  LIFE  SAFETY 

681 

Kivela 

MAXWELL  AFB,  AL 

VISITING  OFFICERS  QRTRS 

894 

Udall 

MAXWEIi  AFB,  AL 

ADAL  SECURITY  POLICE 

2,098 

Udall 

MAXWELL  AFB,  AL 

COMM  GROUP 

4,800 

UdaU 

MAXWELL  AFB,  AL 

STUDENT  DORMITORY 

4,822 

Udall 

MAXWELL  AFB,  AL 

JAG  ACADEMIC  FACILITY 

7,029 

Udall 

MCCONNELAFB,KS 

COMP  HEALTH  CARE  CLINIC 

17,815 

Cole 

MCGUIRE  AFB,  NJ 

ADAL  OPERATIONSnfRAINING 

883 

Moritz 

PATRICK  AFB,  FL 

DINING  FACILITY 

3,795 

Hodges 

SCOTT  AFB,  IL 

HQ  US  TRANSPORTATION 

15,200 

Steck 

SHEPPARD  AFB,  TX 

STUDENT  ENUSTED  HSNG 

8,795 

Steck 

TRAVIS  AFB,  CA 

ARMED  SVCS  BLOOD  FACILITY 

2,029 

Cole 

YOUNGSTOWN,  OH 

ADD  AVIONICS  MAINTENANCE 

1,194 

Koch 
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3.6  Data  Collection 

Data  was  collected  with  a  project  manager  survey,  and  from  compiled  cost  and 
schedule  information.  A  project  manager  interview  was  desired  and  conducted  for 
these  projects  ovw  a  period  of  about  12  weeks  be^nning  in  the  summer  of  1993,  to 
collect  subjective  project  data  for  both  output  measures  and  input  &ctors.  The  survey 
is  included  in  the  appendix.  Both  input  and  output  data,  including  a  subjective 
evaluation  of  cost  and  schedule  goal  achievement,  were  collected  from  project 
managers.  Managers  were  asked  to  rank  various  aspects  of  project  effort  and 
performance  from  1  to  S.  The  researcher  later  converted  these  rankings  into  scores 
for  the  various  input  factors  and  output  measures.  Project  managers  were  individually 
surveyed  via  telephone  or  facsimile  and  the  survey  was  distributed  to  individual  project 
managers  through  Major  Tanya  Banks,  the  point  of  contact  at  AFCEE  at  Brooks 
AFB,  Texas.  Projects  were  first  selected,  and  then  data  were  either  accepted  or 
rejected  depending  on  the  manager's  involvement  with  the  project.  For  the  best  data, 
managers  were  kno^edge^le  about  project  planning,  deagn,  and  construction  efforts 
and  were  also  in  a  good  position  to  judge  project  success.  The  major  drawback  in 
collecting  both  input  and  output  data  from  one  source  was  the  risk  of  introducing  bias. 
A  project  which  turned  out  badly  could  be  blamed  on  inadequate  accomplishment  of 
input  &ctors.  Separate  sources  would  reduce  this  sort  of  bias. 

Cost  and  schedule  data  were  compiled  from  the  AFCEE  Programming,  Design,  and 
Construction  (PDC)  database.  These  data  came  from  the  "Key  Construction 
Information"  screen,  which  included  the  contract  amount  at  award,  and  the 
construction  current  woildng  estimate,  which  was  used  to  calculate  the  cost  variation 
score  (discussed  in  Chapter  4),  as  well  as  the  "contract  days"  data,  the  beneficial 
occupancy  dates  (estimated  and  actqal),  and  the  construction  completion  data,  which 
was  used  to  calculate  the  schedule  performance  score  (discussed  in  Chapter  4).  Some 
PDC  data  were,  however,  ^fficult  to  incorporate  into  this  project.  The  original 
construction  completion  date,  for  example,  was  usually  revised  and  it  was  difficult  to 
identify  what  part  of  the  change  was  due  to  "normal"  schedule  growth  and  what 
portion  was  due  to  influences  outside  the  project.  Although  subjective  cost  and 
schedule  data  were  compiled  from  the  project  manager  questionnaire,  only  the 
objective  data  from  PDC  were  used  to  determine  cost  growth  and  schedule  growth 
scores.  To  convert  the  raw  information  into  output  measures,  a  table  was  devised 
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with  ranges  of  raw  scores  broken  down  into  score  values.  Raw  cost  and  schedule  data 
conversion  information  is  shown  at  Appendixes  B  and  C. 


3.7  Objectives  Matrix  Concept 

A  major  challenge  of  a  metrics  ^em  is  to  measure  a  unique  project  with  multiple 
criteria  of  different  weights,  and  compare  the  results  to  those  of  differoit  projects 
which  may  have  different  objectives  and  criteria.  This  has  been  made  possible  with  the 
application  of  the  objectives  matrix.  The  objectives  matrix  allows  the  researcher  to 
gather  both  objective  and  subjective  data  and  combine  it  to  produce  a  single  index 
which  can  be  compared  to  indexes  of  other  projects  (CD  Publication  S-1  86).  The 
matrix  consists  of  performance  criteria,  a  performance  scale,  weights,  and  the 
performance  index.  Performance  criteria  define  what  is  to  be  measured.  Weights 
determine  the  rdative  importance  of  the  criteria  to  each  otho-  in  the  overall 
measurement  objective.  Criteria  and  wdghts  must  be  valid  for  a  nudrix  to  be 
worthwhile.  Th^  can  be  proposed  by  a  panel  of  experts,  or  a  larger  survey  of 
practicing  professionals,  but  they  must  be  borne  out  by  statistical  research.  The 
performance  scale  compares  the  measured  value  to  a  standard  or  selected  benchmark 
value.  The  performance  index  is  calculated  by  combining  each  criterion's  performance 
score  multiplied  by  its  weight.  The  performance  index  is  used  to  indicate  and  track 
performance.  Example  objectives  matrixes  for  measures  and  input  fectors  are  shown 
in  Figure  3. 1  and  3.2,  respectively. 


INDEX 

785 


4.  Presentation  and  Analysis  of  Data 


4.1  Overview 

TMs  chapter  vnll  present  project  data,  including  input  and  output  indexes,  discuss 
output  index  sen^vity  to  changes  in  output  criteria  wdghting,  compare  input  and 
output  indexes  for  each  project,  and  discuss  data  regression  analysis. 

4.2  Input  Indexes 

The  AFCEE  panel  (see  Table  3.1)  conasting  of  experienced  construction  project 
managers  was  surveyed  to  determine  input  fiictors  and  provide  factor  weights.  Panel 
members  were  given  a  proposed  list  of  factors  and  asked  to  weight  factors  on  a  100 
point  scale  and  also  to  identify  other  posable  factors.  The  panel  determined  the 
following  factors  and  weights  for  all  projects; 


Input  Factor 

Weight 

design  basis 

30 

qualified  key  leaders 

25 

compatible  objectives 

15 

wdl  defined  responsibilities 

15 

control  systems 

10 

project  environment 

5 

No  additional  tiictors  were  identified.  A  description  of  these  factors  is  included  in 
chapter  3.  The  same  wdghts  were  used  for  all  projects  to  more  equitably  compare  the 
levd  of  effort  which  went  into  accomplishing  the  project.  Arguably,  weights  could 
have  beoi  dumged,  depending  on  the  nature  of  the  project,  but  these  factors  and 
weights  were  viewed  as  good  universal  indicators  of  project  effort. 

Input  factor  scores  were  determined  fi'om  project  manager  responses  from  the 
project  manager  interview  (see  appendix).  The  design  basis  factor  was  calculated  by 
avera^ng  the  Project  Definition  and  Design  Execution  sections  of  the  interview.  The 
qualified  key  leaders  score  was  determined  fi'om  responses  to  the  Organization  section 
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of  the  interview.  Specifically,  the  "skills  needed  to  carry  out  their  responsibilities," 
"key  positions  changed,"  and  "supported  by  management"  questions.  The  compatible 
objectives  rating  was  obUuned  fi'om  the  Project  Definition  section  of  the  interview. 
The  well  defined  responsibilities  rating  was  taken  from  the  "roles  and  responsibilities" 
question  in  the  Organization  section.  The  control  systems  score  was  obtained  from 
the  "project  controls  were  effective"  question  in  the  Construction  Execution  section. 
Finally,  the  project  environment  score  came  from  the  overall  evaluation  in  the 
Organization  section. 

Input  indexes  were  calculated  with  an  objectives  matrix  by  multiplying  the  factor 
score  by  the  factor  weight  and  totaling  them  (see  Chapter  3  for  an  explanation  of 
objectives  matrices).  Each  input  score  vm  calculated  by  multiplying  the 
corresponding  interview  result  by  two.  A  factor  score  of  six  was  considered  average, 
since  interview  results  were  based  on  a  five  point  scale  with  three  considered  average. 
Therefore,  an  "average"  index  rating  was  600,  since  factor  weights  totaled  100.  Input 
indexes  ranged  from  a  low  of 320  of  1000  possible  for  the  Alter  Medical  Center  Clinic 
Annex  project  at  Andrews  AFB,  Maryland,  to  a  high  of 970  for  the  Hospital  Life 
Safety  project  at  Little  Rock  AFB,  Arkansas.  Table  4. 1  summarizes  project  input 
index  results. 

4.3  Output  Indexes 

Output  measure  wdghts  were  detomined  by  the  same  AFCEE  panel  of  project 
managers  (see  Table  3.1).  Managers  were  given  a  list  of  proposed  output  measures 
and  asked  to  weight  them  on  a  100  point  scale  and  identify  other  possible  measures. 
The  panel  weighted  output  measures  as  follows: 


Measure 

Weieht 

project  perfomumce 

50 

cost  variability 

20 

schedule  performance 

20 

safety 

10 

Table  4.1:  Input  Results  by  Project 
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Arguably,  weights  could  have  been  changed,  depending  on  the  nature  of  the 
project,  but  these  factors  and  weights  were  viewed  as  good  universal  indicators  of 
results.  Project  performance  scores  were  obtained  from  the  project  manager  surveys. 
Questions  about  how  well  the  project  met  anticipated  requirements,  quality 
management  objectives,  operating  and  maintenance  objectives,  societal  objectives,  and 
the  manager's  opinion  of  the  overall  success  of  the  project  were  used  to  arrive  at  the 
[Project  performance  index.  Cost  variability  was  determined  from  an  index  of  the 
difference  between  the  construction  current  working  estimate  (CWE)  and  the  contract 
amount  at  award,  divided  by  the  contract  amount  at  award  (see  Appendix  B).  These 
data  came  from  the  PDC  database.  Schedule  performance  was  determined  from  an 
index  of  the  revised  contract  days  minus  the  ori^nal  contract  days  scheduled  divided 
by  the  original  contract  days  scheduled  (see  Appendix  C).  Schedule  data  also  came 
from  the  PDC  database.  The  safety  score  was  obtained  from  the  project  manager 
questionnaire  (see  Appendix  A).  Project  managers  were  asked  whether  project  safety 
objectives  were  met  or  exceeded  and  responded  with  their  rating  of  safety 
performance  on  a  scale  of  one  to  five. 

Again,  the  "average"  score  was  considered  as  sbc  with  an  "average"  project  output 
index  of 600  because  the  interview  results  were  on  a  one  to  five  scale,  with  three  as 
the  average,  and  the  objectives  matrix  scores  were  derived  from  interview  responses. 
Scorch  were  calculated  by  multiplying  two  times  the  interview  response.  PDC  cost 
and  schedule  PDC  data  were  converted  to  output  scores  with  a  scale  generated  by  the 
researcher.  Results  of  this  conversion  are  shown  in  .^pendbces  B  and  C.  Output 
indexes  ranged  from  a  low  value  of  526  of  1000  posrible  for  the  Alter  Medical  Center 
Clinic  Annex  project  at  Andrews  AFB,  Maryland,  to  a  high  value  of  952  for  the 
Hospital  Life  Safety  project  at  Little  Rock  AFB,  Arkansas.  It  should  be  pointed  out 
that  the  Little  Rock  project  was  the  smallest  in  the  sample.  Table  4.2  summarizes 
output  results. 

In  general,  sample  project  outputs  were  very  good.  The  average  project 
performance  score  was  8.04.  The  average  cost  score  was  6.86.  The  average  schedule 
score  was  8.04.  The  average  safety  score  was  8.04.  The  average  output  index  value 
for  the  data  was  775.  (A  project  output  index  of 600  was  considered  "average.") 


Table  4.2:  Output  Results  by  Project 
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The  Alter  Medical  Center  Clinic  project  at  Andrews  AFB,  Maryland  had  the  lowest 
quality  rating  of  five.  According  to  the  program  manager,  the  facility  performed  its 
basic  function,  but  many  features  could  have  been  much  better. 

For  this  project  sample,  two  projects  were  rated  "one,"  and  another  rated  "five"  in 
the  cost  criteria  when  the  average  performance  in  this  area  was  6.86.  It  should  be 
noted  that,  although  these  projects  exceeded  their  original  programmed  working 
estimates  and  were  scored  low  for  this  evaluation,  they  did  not  necessarily  exceed  their 
statutory  cost  limits. 

4.4  Output  Sensitivity  Analysis 

The  output  sensitivity  analysis  shows  how  a  change  in  criteria  weight  affects  the 
output  index  for  each  project.  This  analysis  shows  how  an  emphasis  in  a  criterion 
other  than  that  identified  by  the  panel  can  affect  the  overall  output  index  rating,  and 
thus  the  overall  project  "suocess." 

Criteria  weights  were  changed  to  indicate  how  an  emphasis  in  quality,  cost, 
schedule,  or  safety  affected  the  project's  output  index.  Output  criteria  sensitivity  was 
determined  by  increasing  the  measure  in  question  to  SO  (40  for  safety)  and  holding  all 
others  to  20  (10  for  safety)  and  calculating  the  resulting  output  index.  The  "neutral” 
case  was  detemuned  by  holding  all  weights  at  25.  Table  4.3  and  Figure  4. 1  summarize 
conclusions. 

Output  indexes  did  not  change  dramatically  except  for  project  3,  Adal  Power  Plant 
at  Beale  AFB,  California;  project  4,  Upgrade  Emergency  Power  at  Buckley  ANGB, 
Colorado;  and  project  9,  Adal  Operations  Facility  at  Lackland  AFB  Annex,  Texas. 

For  these  projects,  dther  cost  or  schedule  scores,  or  both,  were  significantly  lower 
than  other  output  measures.  Tins  indicates  how  a  poor  showing  in  the  "hard" 
measurement  areas  can  impact  the  project  output  index  when  these  criteria  are 
stressed. 

4.5  Comparison  of  Input  and  Output  Indexes 

Table  4.4  shows  the  comparison  of  input  and  output  indexes  by  project.  The  input 
index  sample  average  was  731,  and  its  standard  deviation  was  149.  The  output  index 
sample  average  was  778,  and  its  standard  deviation  was  105.  If  the  Little  Rock  AFB, 


Table  4.3:  Output  Sensitivity  Results  by  Project 


O  O  Q  O  Q 
OO  OO  S  CM  O 
lo  h*  tf)  o> 


O  O  O  Q  O  O 
(Q  CM  a  O  ^  CM 

o  00  S  A  a> 


OOQQQOOQOO 

OOCMOOWCMOOOtOtO 

i^(x>oooorwo>(or^oou> 


OOQOOQQO 

CM'«“O>r“t^Oic500 

r^(Oi^rooo<oc>co 


Q  O  Q  O  O 
ra  00  CO  ^  CM 


OOQQOOOOOO 
_  C.O>COeOC-CMr*-<OCM<D 

0>ooooN>oooo(Da>r>«.i^a)® 


8  0  0  0 
l5  CM  ^ 
N-  lO  U5  CO 


o  o  o  o 

SCM  <0 
OO  C>  CO 


800Q0000Q0Q 
T-CMCO^OO’r-oiOO) 
00  00  00  00^.^.01^1^00(0 


OOOOOOOOQOO 

Sm(OCM^CMO>CMOOO 
U)N.U)OOh<.C«OOI^OQO 


S  O  o  o  S 

h.  OO  00  00  Co 


s- 

C  oinmmoioioQOioo 

•S^Cots.cMU>r>o|s.ou)ro»o 
«  SCoU)ro^(X}(0Q000(00> 


8ir>  tf)  lA 
CM  Co  Co 

c.  Co  oo  (O 


»-  >-  g 
zoo 


0^  S  o 

>  UJ  I- 

^  iy 

.J  0-  Q 

<Od 
Q  0-  h- 
O  eo 


^  (O 

Ij  01 

U.  UJ  ® 

2  <  Ct 

gcoiu 

gmo 

Ui  -J  o 
^<0 
obz 

< 

Q  O  SS 
<x> 


(0 

a: 

til 

q:  5 

lU  o 
o  S 

Hi 


o 

0. 

§  o 

O  01 

til  o 

-I 

§o 
<  o 


oo 

2  o 

O  Ul 

H  9 
z  ^ 
So 

Q  < 

3  o 

I- 

CO  ^ 


0 
2Z 
S  z 

Sgf: 

0\ 

i^i 

^  S  5 

o  o  = 

O  <  D 


U.  LU 
§1 
Ss 

2z 

Q  ^ 

uj  ^ 

SO 
01  o 
<  < 


Si:  iL  V.  ^ 

^  <  til  01 

S  lii  ^  H 

iss§ 

<  00  m  0 


§1^1 
<  ^  o  < 

q9o-j 
z  5  01  -J 

$  <  til 
5  J  til  5 

ills 


^5^  - 

.  .  -  CD 
00  00  >4 
U.  LL  < 
<  <  -I 
-I  .U  Ui 
-I  Z 

UJ  til  z 

11° 


eg 
<  <0 
CO  0 

p 


_  ©▼-CMCO'M-IOCOCoOO<J)Ot-CM 

CMCO^lAtOCoCOWT-^T-T-^r-T-T-T-T-CMCMCM 


Mean  768  775  740  767  775 

Standard  Deviation  116  107  139  146  113 


Table  4.4:  Comparison  of  Input  and  Output  Indexes  by  Project 
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Arkansas  Hospital  Life  Safety  project,  the  snuillest  and  probably  least  significant 
project  for  this  research,  is  neglected,  the  input  index  sample  average  is  720,  its 
standard  deviation  is  142;  the  output  index  sample  average  is  770,  and  its  standard 
deviation  is  100.  These  numbers  indicate  a  much  higher  than  average  rating  for  both 
inputs  and  outputs  if  the  "average”  project  is  assumed  to  rate  500  on  both  rating 
scales.  The  absolute  difference  column  ^ves  a  relative  idea  of  how  well  input  indexes 
and  output  indexes  agree.  Some  large  differoices,  the  Andrews  AFB,  Maryland  Alter 
Medical  Center  Clinic  Annex  project;  the  Lackland  AFB  Annex,  Texas  Adal 
Operations  Facility  project;  the  Maxwell  AFB,  Alabama  JAG  Academic  Facility 
project;  the  McGuire  AFB,  New  Jersey  Adal  Operations  and  Training  project;  and  the 
Scott  AFB,  Biinois  HQ  U.S.  Transportation  project  absolute  differences  suggest  some 
possible  environmental  factors  not  taken  into  account  by  the  questionnaire.  The  input 
indexes,  except  for  the  Andrews  AFB  project,  are  significantly  higher  than  output 
indexes  for  these  projects,  indicating  that  factors  other  than  those  assumed  to  be 
critical  affected  the  output  rating.  This  can  be  partially  explained  by  the  way  index 
criteria  were  weighted  for  both  input  and  output  indexes.  The  sensitivity  analysis 
section  ^owed  the  impact  of  criteria  on  index  values. 


4.6  Regression  Analysb 

A  linear  regression  analyas  was  accomplished  with  Microsoft  Excel  statistical 
analysis  software.  Regression  uses  the  "least  squares"  analysis  method  to  fit  a  line 
through  a  set  of  observations.  Regression  is  used  to  analyze  how  a  dependent  variable 
is  affected  by  one  or  more  independent  variables.  In  this  case,  it  is  to  determine  how 
one  independent  variable,  the  input  index,  affects  the  dependent  variable,  the  output 
index.  Even  though  such  an  analysis  may  indicate  a  statistical  relationship  between 
variables,  it  does  not  necessarily  establish  a  cause-and-effect  relationship. 

Regression  analysis  results  for  the  "quality  driven"  project  are  sununarized  in  Table 
4.5.  The  corresponding  line  of  best  fit  graph  is  shown  in  Figure  4.2.  Correlation 
coeflScients  for  the  other  project  "types"  (see  the  Output  Sensitivity  Analysis  section) 
ranged  firom  a  low  of  0.5819  for  the  "cost  driven"  project  to  a  high  of  0.8419  for  the 
"quality  driven"  project.  A  correlation  coefficient  of  zero  indicates  no  correlation. 


while  a  coeffici<mt  of  one  is  a  positive  correlation.  This  suggests  that  these  data  are 
relatively  strongly  positively  correlated. 
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Table  4.5:  Begreaiioa  Aaalysb  Results  for  Quality  Driven  Project 

Regression  Statistics 

Correlaticm  CoefiBdent 

0.841878 

R Square 

0.708758 

Adjusted  R  Square 

0.694196 

Standard  Enor 

59.40587 

Observations 

22 

466.0724 

0.790956 


Coefficient 

StdJnor 

t  .Sta^sric 

Inteioqa 

330.4062 

65.0377 

5.0802 

194.7399 

Input 

0.608897 

0.08728 

6.9765 

0.426838 

5.  Conclusions 


Although  this  investigation  involved  a  relatively  small  number  of  projects,  three 
conclusions  can  be  realized  from  the  analyses.  These  three  conclusions  are  listed 
below. 

1.  Relatively  high  input  and  output  indexes  indicate  good  project  management. 
Input  index  values  averaged  73 1  and  ranged  from  a  low  value  of 320  for  the  Andrews 
AFB,  Maiyland  Alter  Medical  Center  Clinic  Annex  project  to  a  high  value  of  970  for 
the  Little  Rock  AFB,  Arkansas  Hospital  Life  Safety  project.  Output  index  values 
averaged  77S  and  ranged  from  a  low  value  of  520  for  the  Buckley  ANGB,  Colorado 
Upgrade  Emergency  Power  project  to  960  for  the  Little  Rock  AFB,  Aikansas  Hospital 
Life  Safety  project. 

2.  Statistical  analyses  indicate  a  strong  positive  linear  correlation  between  input 
and  output  indexes.  The  input  index/output  index  correlation  coefficient  for  these  data 
was  0.8419  for  the  weights  determined  by  the  AFCEE  panel  (the  "quality"  output 
index).  Other  weighting  combinations  resulted  in  correlation  coefficients  of  0.7426  for 
"neutral"  output  index,  0.5819  for  the  "cost"  (Hitput  index,  and  0.6560  for  the 
"schedule"  output  index.  These  coefficients  suggest  that  input  index  ratings  can 
rdiably  be  used  to  predict  output  measures.  This  analysis  alone  does  not  necessarily 
indicate  a  cause-and-effect  rdationslup. 

3.  A  reasonable  cause-and-efiect  relationship  can  be  shown  by  tracing  the  logic 
used  to  design  the  projea  manager  survey.  Input  factor  statements  were  designed  to 
determine  the  level  of  effort  which  went  into  project  planning,  design  and 
construction.  Measures  of  success  statements  were  designed  to  determine  project 
results.  If  both  input  fitctors  and  output  measures  questions  are  appropriate,  it 
logically  follows  that  a  cause-and-effect  relationdiip  has  been  established.  This 
suggests  that  it  is  possible  to  predict  project  success  based  upon  input  data. 

Therefore,  the  objectives  matrix  evaluation  of  input  factors  would  be  a  good  predictor. 
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6.  Recommendations 


As  with  any  study,  two  types  of  recommendations  have  resulted.  First,  the 
study  indicates  certain  acti>nties  that  are  important  in  project  management  and 
organization.  Second,  further  areas  for  research  have  been  identified. 


6.1  Recommeadations  for  AFCEE  Action 

1.  Key  leaders:  the  project  numager,  construction  manager,  construction 
superintendant,  and  project  en^eer  should  be  assigned  to  the  project  fi’om  start  to 
finish.  The  interruptions,  learning  curve  problems  and  ddays  associated  with  key 
personnel  turnover  on  projects  are  unacceptable.  In  addition,  project  "ownership,"  the 
pride  in  playing  an  integral  part  in  a  project's  success  or  ftihire,  is  lost  each  time  it  is 
handed  off. 

2.  Increased  effort  should  be  spent  in  the  planning  phase  identifying  the  project 
objectives  of  key  participants.  Team  building  and  formal  project  objective  setting  will 
reduce  conflicting  objectives  and  foster  improved  communication. 

3.  Deagn  criteria  and  operational  fiK:tors  should  be  communicated  to  the 
deagner.  Constructability  studies  should  be  included  as  part  of  the  desi^  process. 

4.  Roles  and  responsibilities  should  be  clear.  An  organization  chart  should  be 
developed  and  responsibilities  should  be  specified.  Planning  meetings  and  team 
builcfing  should  be  used  to  clarify  responribilities. 

5.  The  project  information  management  system  should  be  improved.  The  existing 
PDC  systm  rqmrts  cost  and  schedule  informarion,  but  does  little  to  identify  problem 
areas  in  a  timefy  fiishion.  Managers,  especially  the  construction  manager,  need  project 
schedule,  labor  productivity,  equipment  management,  and  materials  management 
information  during  project  accomplishment  to  effectively  control  a  project.  While  this 
should  be  done  by  the  contractor,  too  much  is  at  stake  to  leave  it  to  chance. 
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6.2  Further  Research 

The  following  areas  merit  additional  study: 

1.  The  results  of  nmre  concerted  planning  efforts,  including  setting  formal  project 
objectives,  and  establishing  clear  responsibilities. 

2.  The  feasibility  of  the  design  and  implementation  of  a  new  project  information 
management  ^stem  for  construction  and  project  managers. 

3.  The  results  of  including  a  constructability  study  as  part  of  the  design. 

4.  The  feaability  and  results  of  maintaining  the  same  key  project  leaders  from 
project  start  to  finish. 
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Appendix  A:  Project  Manager  Interview 
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Project 

Name 

Location 


Company 
Fax 

Inter>idew 

Scheduled  Date/Time; 

Actual  Date/Time: 


Interviewee 

Name 

Title 

Phone 


This  interview  should  take  around  20  minutes.  Will  this  be  a  problem? 

Before  we  begin,  Fd  like  to  read  an  introduction  and  answer  any  questions. 

This  interview  is  being  conducted  to  compile  data  for  research  on  inputs  impacting 
project  success.  Fm  studying  aspects  of  project  planning  and  execution  and  their 
impact  on  overall  project  success. 

Your  name  and  was  given  to  me  by  of  as 

the  representative  for  the  project.  For  this  interview,  Fd  like  to  know  your  opinion  on 
the  effort  ^)ent  planning  and  executing  the  project  and  it's  ultimate  success.  Do  you 
feel  qualified  and  comfortable  answering  these  questions?  (If  not,  someone  else?). 

The  intennew  is  divided  into  two  sections,  the  first  about  project  success,  the 
second  about  planning  and  execution  efforts.  Fli  read  a  list  of  statements  about  the 
project.  Fll  ask  you  to  provide  a  response  on  a  scale  of  1  to  5,  with  1  meaning  you 
strongly  disagree,  and  S  meaning  you  strongly  agree.  If  you  have  no  opinion,  please 
state  so. 

Your  responses  will  be  used  only  for  this  research. 

Do  you  have  questions  before  we  be^n? 
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Measures  of  Success 

These  questions  are  about  the  project  results.  Please  use  a  1  to  5  scale  with  S 
meaning  strong  agreement  and  1  meaning  disagreement  to  answer. 

1.  Project  met  requirements  (objectives).  Please  comment. 

2.  Quality  objectives. 

a.  In  general,  project  participants  worked  and  communicated  well  together 
(owners,  customers,  contractors,  managers). 

b.  The  project  was  characterized  by  competent,  consistent  management. 

c.  Emphasis  was  placed  on  satisfying  the  needs  of  the  customer. 

d.  Comments: 

3.  Project  control  objectives. 

a.  Budget  objectives  were  met  or  exceeded. 

b.  Schedule  objectives  were  met  or  exceeded. 

c.  Quality  objectives  were  met  or  exceeded. 

d.  Safety  objectives  were  met  or  exceeded. 

e.  Comments: 

4.  Planning,  and  design  objectives. 

a.  Objectives  were  weQ  established  and  agreed  upon,  especially  by  client/owner. 

b.  Well  defined  scope  contributed  to  smooth,  successful  project  execution. 
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c.  Constructability  and  maintainability  assessments  and  lessons  learned  from  other 
projects  were  incorporated  into  the  engineering  and  design  phase. 

d.  Actual  project  execution  (method,  controls,  schedule,  etc.)  matched,  or  nearly 
matched  planned  project  execution  at  authorization. 

e.  Comments: 

5.  Societal  objectives. 

a.  Regulatory  requirements  were  met  or  exceeded. 

b.  Good  labor  relationship  was  maintained  during  construction. 

c.  Safety  and  health  goals  were  met  or  exceeded  during  execution. 

d.  Project  met  EEO  goals  (minority  business,  disadvantages 

e.  Project  met  or  exceeded  community  relations  goals. 

f.  Education  and  training  goals  were  met. 

g.  Comments: 

6.  Operating  and  Maintenance  objectives  (including  ease  of  transition). 

a.  Smooth  transition  from  construction  to  operation. 

b.  Operational  and  nuunten-nce  requirements  were  met  or  exceeded. 

c.  No  major  renovation  was  needed  to  meet  operational  or  maintenance 
requirements  existing  when  project  was  authorized. 


d.  Comments: 
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7.  What  is  your  overall  rating  of  the  success  of  this  project?  What  are  your  main 
reasons  for  your  assessment? 

What,  if  anything,  could  be  improved  to  make  the  next  project  more  successful? 

Input  Factors 

The  next  questions  are  about  the  planning,  de^gn,  and  construction  effort.  Please 
answer  the  following  questions  on  the  same  1  to  5  scale. 

8.  Project  definition. 

a.  Project  objectives  were  clear,  understood,  and  agreed  upon  by  all  parties. 

b.  Project  scope  was  well  defined  at  the  time  the  project  was  authorized. 

c.  Project  risks  were  identified  and  evaluated  during  project  planning. 

d.  Project  alternatives  were  sufficiently  identified  and  evaluated;  alternative 
selection  was  made  by  appropriate  authority. 

e.  Project  execution  plan  was  well  defined  during  planning  stage,  (project 
strategy,  i.e.  contracting  approach,  controls  systems,  scheduling,  etc.) 

f.  Comments: 

9.  Organization. 

a.  Roles  and  responsibilities  were  cleariy  outlined  and  understood,  (especially 
those  of  key  players) 

b.  Team  members  had  the  skills  needed  to  carry  out  their  responsibilities. 
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c.  In  general,  individuals  on  the  project  team  worked  well  together. 

d.  People  in  key  positions  changed  frequently  (project  executive,  project 
manager,  general  superintendent). 

e.  Teams  were  supported  by  management. 

f.  Comments: 


10.  Design  execution. 

a.  Design  documents  were  accurate  (drawings  and  specifications  with  few  or  no 
changes  or  corrections). 

b.  Design  documents  were  usable  (drawings  and  specs  complete  and  clear  to 
those  using  them). 

c.  Design  was  constructable.  (Project  design  integrated  construction  knowledge 
into  planning,  engineering,  procurement,  and  field  operations  to  accomplish  project 
objectives.) 

d.  Design  was  cost  effective.  (Design  costs,  including  approved  changes,  fell 
within  design  budget.) 

e.  Design  was  efficient,  (efficient  sizing  of  members,  efficient  layout,  appropriate 
tolerances,  etc.  No  "gold  plating.") 

f  Design  and  procured  materials  were  delivered  within  schedule. 

g.  Project  start  up  was  vnthout  significant  problems  (attributable  to  design). 

h.  Comments: 
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11.  Construction  execution. 

a.  Project  controls  were  effective,  (problems  identified  early,  corrective  action 
triggered,  e.g.  cost,  schedule,  quality,  safety,  materials,  labor  productivity,  equipment, 
etc.) 

b.  Qualified,  effective  construction  management. 

c.  Constructors  and  craftsmen  were  qualified,  motivated,  and  effective. 

d.  Construction  methods  were  effective. 

e.  Effective  superviuon  and  field  planning. 

f.  Effective  communication. 

g.  Minimum  construction  interference. 

h.  Minimum  changes  and  rework. 

i.  Comments: 

12.  What  is  your  overall  opinion  of  the  effort  put  into  the  planning,  design,  and 
construction  of  this  project?  What  are  your  main  reasons  for  this  assessment? 


What,  if  anything,  could  have  been  done  to  improve  this  effort? 


Converting  Raw  Cost  Variation  to  Cost  Score 
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Appendix  B:  Raw  Cost  Data  Conversion 
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Appendix  C:  Raw  Schedule  Data  Conversion 
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